Permutation Tests and Confidence Intervals for the Area under the Roc-curve

We investigate rank-based studentized permutation methods for the nonparametric Behrens-Fisher problem, i.e. inference methods for the area under the ROC-curve (AUC). We hereby prove that the studentized permutation distribution of the Brunner-Munzel rank statistic is asymptotically standard normal, even under the alternative. This does not only imply consistency of the corresponding permutation test, but also that confidence intervals for the underlying treatment effects can be computed. The result further implies that the Neubert and Brunner studentized permutation test can be inverted for the computation of confidence intervals. In addition, we derive permutation-based range-preserving confidence intervals. Extensive simulation studies show that the permutation based confidence intervals appear to maintain the pre-assigned coverage probability quite accurately (even for rather small sample sizes). For a convenient application of the proposed methods, a freely available software package for the statistical software R has been developed. A real data example illustrates the application.